Skip to content

Fix: run the orphaned unit:integrations vitest project from the root test scripts - #236

Open
AmaadMartin wants to merge 7 commits into
mainfrom
fix/run-unit-integrations-vitest-project
Open

Fix: run the orphaned unit:integrations vitest project from the root test scripts#236
AmaadMartin wants to merge 7 commits into
mainfrom
fix/run-unit-integrations-vitest-project

Conversation

@AmaadMartin

@AmaadMartin AmaadMartin commented Jul 29, 2026

Copy link
Copy Markdown
Owner

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

  1. Link to an existing issue (if applicable):

No existing issue. The gap originates in google#449, the PR that added the integrations
package together with the unit:integrations vitest project.

  1. Or, if no issue exists, describe the change:

Problem: vitest.config.ts declares a unit:integrations project that owns
integrations/test/**/*_test.ts, but no root script and no workflow ever selected it — the string
unit:integrations appeared exactly once in the whole repo, in its own declaration. The three
aggregate scripts in package.json enumerate projects by hand and all three omitted it:

"test":          "vitest --project unit:core --project unit:dev --project integration --project e2e",
"test:unit":     "vitest --project unit:core --project unit:dev",
"test:coverage": "vitest run --project unit:core --project unit:dev --project integration --project e2e --coverage",

.github/workflows/validation.yaml runs only npm run test:coverage, so that workspace's tests
had never executed anywhere — not in CI, and not for a contributor following
CONTRIBUTING.md. This is config drift from google#449, which added the project and the
workspace but did not touch the --project enumerations.

The dormant suite had already rotted. integrations/test/version_test.ts asserted
expect(version).toBe('1.3.0') while integrations/src/version.ts had moved on to '1.4.0' at
this branch's base — so simply wiring the project in would have turned CI deterministically red.
The two halves have to ship together.

Solution: two edits.

  1. package.json — add --project unit:integrations to test, test:unit and
    test:coverage, positioned after unit:dev so the order mirrors vitest.config.ts.
    test:unit is included deliberately: it is a unit project, and leaving it out of the script
    contributors reach for most would recreate the same drift. cross-language stays out of all
    three — it has its own workflow (.github/workflows/cross-language-integration.yml) and needs
    a Go toolchain.

  2. integrations/test/version_test.ts — assert the exported constant against
    integrations/package.json instead of a literal:

    expect(version).toBe(packageJson.version);

    Why this and not just retargeting the literal: release-please rewrites
    integrations/package.json (release-type: node) and integrations/src/version.ts (an
    extra-files entry) in the same commit, but never touches test files. A retargeted literal
    would be stale again on the next release, and the failing check would block the release PR
    itself. Anchoring to the manifest pins the invariant that actually matters — the exported
    constant equals the published package version — and is self-maintaining. This is not
    hypothetical: main has since moved to 1.5.0, and the derived assertion holds there with no
    test edit.

    This is a rewrite of an existing test rather than an addition, which the repo guideline
    normally discourages. It is the documented exception — the old assertion encoded a value that
    was already wrong — and it is in its own commit (cc69c3e6e), separate from the
    package.json change (a2c057a8b), so the assertion change can be reviewed on its own.

How the manifest is read, and why not readFileSync. The manifest is pulled in with a native
JSON module import, import packageJson from '../package.json' with {type: 'json'}, rather than
JSON.parse(readFileSync(new URL('../package.json', import.meta.url), 'utf-8')). The usual
argument for the readFileSync form is that a JSON import needs resolveJsonModule, which the
root tsconfig.json does not set — but that argument does not hold in this repo, and it was
checked rather than assumed:

$ npx tsc --showConfig -p tsconfig.json | grep -E 'module|resolveJsonModule'
        "module": "nodenext",
        "moduleResolution": "nodenext",
        "resolveJsonModule": true,

resolveJsonModule is already on, implied by "module": "nodenext", so no tsconfig.json change
is needed and no compiler behaviour changes for core, dev or integrations. npx tsc --noEmit
reports zero errors under integrations/. The import form is also the established pattern
here — 20 existing occurrences across tests/integration/** — and it yields a typed
packageJson.version directly, where the readFileSync form needs a hand-written
{version: string} annotation over a JSON.parse that actually returns any. Fewer lines, no
new tsconfig surface, better typing, existing convention.

vitest.config.ts is intentionally unchanged. Adding a project can only add covered lines;
coverage.include is untouched so the denominator does not move, and integrations/src/** was
already in it at 0% via coverage.all. Coverage is strictly non-decreasing and the
86/87/88/86 thresholds are unaffected — CI confirmed every metric rose on every leg (ubuntu
90.68/89.61/91.55/90.68 → 90.69/89.62/91.63/90.69). The threshold ratchet is filed separately.

Out of scope, deliberately: a guard preventing this drift class from recurring (queued as its
own task), and coverage for integrations/src/index_web.ts.

Rejected alternatives.

  1. Delete the unit:integrations project as intentionally dormant. The evidence says oversight,
    not intent: feat(integrations): create new top-level integrations package google/adk-js#449 wired up every other integration point (workspace entry, project
    definition, the alias on all six projects, coverage.include, the release-please entry) and
    missed only the two scripts lines. It also carries none of the three markers that make
    cross-language's exclusion deliberate — no dedicated script, no dedicated workflow, no
    toolchain prerequisite. @google/adk-integrations is a published, release-managed package, so
    deleting the project just re-arms the same silent-no-run trap for the next contributor.
  2. Keep the literal and bump it to the current version. Covered above: it re-creates the bug at
    the next release, and the failing check would land on the release PR itself.
  3. Add the test to release-please extra-files with an x-release-please-version annotation.
    That makes the test tautological — automation would write both sides from one value, so it
    could never detect a desync — and extends release automation's write surface into test code.
  4. Replace the explicit project list with an exclusion filter (--project '!cross-language').
    The pinned vitest supports it, and it would structurally prevent this class of drift. Rejected
    here because validation.yaml runs the matrix on ubuntu-latest, windows-latest and
    macos-latest, and a ! inside an npm script argument is a quoting hazard across sh and
    cmd.exe. Not worth that risk in the single command that gates every PR, against a four-word
    explicit edit. Worth revisiting on its own.

Review history — one test was added and then removed. An earlier revision of this PR added
integrations/test/index_web_test.ts, asserting that the web entry point exports exactly the node
surface. It has been removed (05cfab7f7). It was out of scope, and its stated rationale was
wrong for this codebase: core/src/index_web.ts re-exports only ./common.js against a 60-line
core/src/index.ts, because node-only symbols (GcsArtifactService, DatabaseSessionService,
UnsafeLocalCodeExecutor) must not reach a browser bundle. The two surfaces are designed to
diverge, and there is no test of that kind anywhere in the repo. The first node-only export added
to integrations would have failed the test and pressured the wrong fix. Nothing the plan asked
for lost coverage: index_web.ts returns to its pre-existing 0%, which the plan explicitly
declared out of scope.

Collision check: gh pr list --repo AmaadMartin/adk-js --state open --limit 1000 over all 531
open PRs, plus gh pr diff --name-only on every adjacent hit. Findings:

Testing Plan

Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.

Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.

Baseline — the dead project's failure. npx vitest run --project unit:integrations before the
fix, which is the evidence the project was dead:

FAIL  |unit:integrations| integrations/test/version_test.ts > version > should return the correct version
AssertionError: expected '1.4.0' to be '1.3.0' // Object.is equality

After the fix:

✓ |unit:integrations| integrations/test/version_test.ts (1 test) 5ms
 Test Files  1 passed (1)
      Tests  1 passed (1)

Mutation proof 1 — the test can actually fail (desync detection). A version test comparing two
values that are equal by construction would carry no signal, so the assertion was proved live by
mutating the source it pins. integrations/src/version.ts temporarily set to '1.4.1', leaving
integrations/package.json at 1.4.0:

× version > should match the version declared in package.json 13ms
  → expected '1.4.1' to be '1.4.0' // Object.is equality
 Test Files  1 failed (1)

Mutation proof 2 — the test survives a release bump (self-maintenance). This is the property
the old hardcoded literal lacked, so it is proved too. Simulating a release-please commit by
setting both integrations/src/version.ts and integrations/package.json's version to
1.6.0:

✓ |unit:integrations| integrations/test/version_test.ts (1 test) 6ms
 Test Files  1 passed (1)

Both mutations reverted immediately; git status --porcelain is empty and neither
integrations/src/version.ts nor integrations/package.json is in the diff (both are
release-please-owned and must never be hand-edited).

Wiring proof — positive and negative. npx vitest list --project unit:core --project unit:dev --project unit:integrations (the exact project set of the fixed test:unit) collects the
integrations/test entries; the same command with main's two-project selector collects zero.

Static gates on the touched files: npx eslint integrations/test/version_test.ts clean;
npx prettier --check integrations/test/version_test.ts package.json clean. npx tsc --noEmit
reports only pre-existing errors under core/test/** and tests/** — zero under integrations/,
and the changed test file is confirmed present in the tsc program.

Coverage-gate run — the check that actually proves this change. It exercises the threshold gate
on the exact projects the fix adds, and is a strict subset of the CI command:

$ env -u GOOGLE_CLOUD_PROJECT npx vitest run \
    --project unit:core --project unit:dev --project unit:integrations --coverage
File               | % Stmts | % Branch | % Funcs | % Lines |
 integrations/src   |   66.66 |        0 |       0 |   66.66 |
  index.ts          |     100 |      100 |     100 |     100 |
  index_web.ts      |       0 |        0 |       0 |       0 | 1-7
  version.ts        |     100 |      100 |     100 |     100 |
exit 0   (no threshold error)

index.ts and version.ts go 0% → 100%. index_web.ts stays at 0%: it is a browser-entry
re-export nothing imports. That is left alone deliberately — no contrived test for it, and no
coverage-ignore pragma, since the repo guidelines class those as suppressions.

Manual End-to-End (E2E) Tests:

  1. npm install && npm run build. The build is required: tests/global_setup.ts imports
    @google/adk with no alias in scope, so it resolves through core/dist. Without it every
    vitest invocation dies with Failed to resolve entry for package "@google/adk" and
    misleadingly reports No test files found. This is pre-existing behaviour.
  2. npm run test:coverage — the only script CI runs.
  3. Confirm the run summary now contains a line tagged |unit:integrations| for
    integrations/test/version_test.ts with a . On main no such line exists and the file is
    absent from the file list entirely.
  4. git diff main --stat lists exactly two files: package.json and
    integrations/test/version_test.ts.

Two local failures on a developer workstation are pre-existing and unrelated, confirmed by
running them on an unmodified checkout:
core/test/code_executors/unsafe_local_code_executor_test.ts and
dev/test/cli/cli_create_test.ts (the gcloud-defaults case). Neither file is touched here.

The authoritative end-to-end signal is CI on this PR: validation.yaml runs
npm run test:coverage on ubuntu, windows and macos, and the unit:integrations project now
appears on all three legs.

Checklist

[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.

Amaad Martin added 2 commits July 29, 2026 22:11
integrations/test/version_test.ts hardcoded `expect(version).toBe('1.3.0')`
while integrations/src/version.ts exports '1.4.0', so the test is currently
red. It went stale unnoticed because the vitest project that owns it
(`unit:integrations`) is invoked by no npm script and no workflow, so the
file has never actually run.

Bumping the literal to '1.4.0' would only re-rot on the next release: the
release automation rewrites integrations/src/version.ts (via the
x-release-please-version annotation) but never touches test literals. Assert
against the version declared in integrations/package.json instead. Both
sides are updated in the same release commit, so the assertion is
self-maintaining, and it guards the invariant actually worth guarding: the
exported constant must not drift from the published package version.
vitest.config.ts declares a `unit:integrations` project owning
integrations/test/**/*_test.ts, but the name appeared nowhere else in the
repository: no npm script and no workflow invoked it, so those tests never
ran. validation.yaml runs `npm run test:coverage` and
cross-language-integration.yml runs `npm run test:cross-language`, which
between them reached every project except this one.

Add `--project unit:integrations` to `test`, `test:unit` and
`test:coverage`, positioned after `unit:dev` to match the declaration order
in vitest.config.ts.

No `test:integrations` script is added on purpose: it would sit one
character from the existing `test:integration` (which runs the unrelated
`integration` project over tests/integration/) and invite mistakes.
`unit:core` and `unit:dev` have no individual scripts either.

The coverage thresholds are deliberately left untouched. coverage.include
already lists integrations/src/**/*.ts and coverage.all defaults to true,
so those files were already in the denominator scored at 0%; running the
project only adds to the numerator. Measured over `unit:core + unit:dev`
(v8), All files goes 88.94/88.11/89.58/88.94 to 88.95/88.14/89.73/88.95
statements/branches/functions/lines - every metric up.
… projects

The two project names differ by one character and by scope: unit:integrations
owns the integrations/ workspace package, integration owns the cross-component
suite in tests/integration/. That similarity is part of why the former was
overlooked by the root test scripts for three releases. Comments only; no
project definition, glob, or threshold changes.
@AmaadMartin

Copy link
Copy Markdown
Owner Author

Independent verification from a duplicate task that was elaborated against the same three defects and is being closed in favour of this PR. Recording the measurements here so they are not lost — all run locally against this branch's exact two-file state (npm install + npm run build first, since tests/global_setup.ts imports @google/adk through core/dist).

Postconditions

check result
npx vitest run --project unit:integrations Test Files 1 passed (1) / Tests 1 passed (1)
npm run test:unit -- --run selects it ✓ |unit:integrations| integrations/test/version_test.ts (1 test)

The only failure in the test:unit run was dev/test/cli/cli_create_test.ts > "Vertex AI selection with gcloud defaults", a pre-existing sandbox failure (no gcloud config), unrelated to this change.

Mutation proof, both drift directions — the assertion fails when it should:

  • forward, integrations/src/version.ts'9.9.9':
    AssertionError: expected '9.9.9' to be '1.5.0' // Object.is equality at integrations/test/version_test.ts:13:21
  • reverse, integrations/package.json1.6.0, version.ts left at 1.5.0:
    AssertionError: expected '1.5.0' to be '1.6.0' // Object.is equality

Anti-rot — bumping both integrations/package.json and integrations/src/version.ts to 1.6.0 keeps it green with no test edit. That is the property the change exists to provide, confirmed.

Two things a reviewer might otherwise ask for, which the evidence says not to:

  1. An explicit expect(packageJson.version).toMatch(/^\d+\.\d+\.\d+/) vacuity guard. Unreachable here. I deleted the version field from integrations/package.json entirely and the test still failed loudly: AssertionError: expected '1.5.0' to be undefined. The imported version is a string literal that can never itself be undefined, so the vacuous undefined-vs-undefined comparison such a guard defends against cannot occur. The guard only makes sense against a readFileSync + JSON.parse form typed {version?: string}; the import attribute used here removes the failure mode rather than asserting against it. Adding it would be dead code.
  2. Whether import ... with {type: 'json'} breaks npm run ts:check. It does not. integrations/test/version_test.ts is in the tsc program (confirmed via tsc --noEmit --listFiles) and npx tsc --noEmit reports zero errors for it — identical to baseline, with no tsconfig.json edit required.

Amaad Martin added 2 commits August 1, 2026 13:27
The disambiguation between unit:integrations and integration was written
twice, once from each side, and each copy restated the include glob two
lines below it. Keep the comment on unit:integrations -- the project this
change wires in, and the less obvious of the two -- and drop the mirror.
The approved spec scopes this change to the three root test-script strings
in package.json and the integrations version test, touching vitest.config.ts
"at most" for the coverage thresholds block -- whose expected outcome is no
edit at all. The project-naming comment added earlier sits outside that
ceiling, so it is removed and the diff is now exactly the two files the fix
requires. No behaviour change: the comment never affected project resolution.
integrations/build.js compiles src/index.ts and src/index_web.ts into
separate published artifacts -- dist/esm and dist/cjs from the first, the
dist/web bundle the package's browser field points at from the second --
so an export added to one entry point and forgotten in the other ships a
browser bundle silently missing the symbol. Nothing guarded that, and
index_web.ts was the one integrations source file no test reached, sitting
at 0% coverage even after the project was wired into the root scripts.

Assert the two entry points expose the same export names and the same
version binding. The key-set assertion is guarded against passing
vacuously if both entry points ever resolve to nothing.
The plan scoped this change to three edits and named index_web.ts coverage
as out of scope. The test also asserted the wrong invariant: it required the
web entry point to export exactly the node surface, but core/src/index_web.ts
re-exports only ./common.js against a 60-line core/src/index.ts, because
node-only symbols such as GcsArtifactService and UnsafeLocalCodeExecutor must
not reach a browser bundle. The surfaces are meant to diverge, so the first
node-only export added to integrations would have failed the test and pushed
the wrong fix.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant